Towards algebraic methods for maximum entropy estimation

نویسنده

  • Ambedkar Dukkipati
چکیده

We show that various formulations (e.g., dual and Kullback-Csiszar iterations) of estimation of maximum entropy (ME) models can be transformed to solving systems of polynomial equations in several variables for which one can use celebrated Gröbner bases methods. Posing of ME estimation as solving polynomial equations is possible, in the cases where feature functions (sufficient statistic) that provides the information about the underlying random variable in the form of expectations are integer valued.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Modeling of the Maximum Entropy Problem as an Optimal Control Problem and its Application to Pdf Estimation of Electricity Price

In this paper, the continuous optimal control theory is used to model and solve the maximum entropy problem for a continuous random variable. The maximum entropy principle provides a method to obtain least-biased probability density function (Pdf) estimation. In this paper, to find a closed form solution for the maximum entropy problem with any number of moment constraints, the entropy is consi...

متن کامل

On Parametric and Implicit Algebraic Descriptions of Maximum Entropy Models

Main aim of this paper is to present some notions on how results from commutative algebra and algebraic geometry could be used in representation and computation of maximum and minimum entropy (ME) models. We show that various formulations of estimation of ME models can be transformed to solving systems of polynomial equations in cases where an integer valued sufficient statistic exists. We give...

متن کامل

Robust Entropy Rate for Uncertain Sources and its Applications in Controlling Systems Subject to Capacity Constraints

The robust entropy rate is defined as the maximum of the Shannon entropy rate, when the joint probability density function of the source is unknown. The uncertainty of the source probability density is described via a relative entropy constraint set between the uncertain source probability density and the nominal source probability density. For this class of problems, the explicit solution for ...

متن کامل

Fast parameter estimation for joint maximum entropy language models

This paper discusses efficient parameter estimation methods for joint (unconditional) maximum entropy language models such as whole-sentence models. Such models are a sound framework for formalizing arbitrary linguistic knowledge in a consistent manner. It has been shown that general-purpose gradient-based optimization methods are among the most efficient algorithms for estimating parameters of...

متن کامل

Minimum Entropy Estimation of Hierarchical Random Graph Parameters for Character Recognition

2.1 Entropy and mutual information In this paper, we propose a new parameter estimation method called minimum entropy estimation (MEE), which tries to minimize the conditional entropy of the models given the input data. Since there is no assumption in MEE for the correctness of the parameter space of models, MEE will perform not less than the other estimation methods such as maximum likelihood ...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:
  • CoRR

دوره abs/0804.1083  شماره 

صفحات  -

تاریخ انتشار 2008